A Spike and Slab Restricted Boltzmann Machine
نویسندگان
چکیده
We introduce the spike and slab Restricted Boltzmann Machine, characterized by having both a real-valued vector, the slab, and a binary variable, the spike, associated with each unit in the hidden layer. The model possesses some practical properties such as being amenable to Block Gibbs sampling as well as being capable of generating similar latent representations of the data to the recently introduced mean and covariance Restricted Boltzmann Machine. We illustrate how the spike and slab Restricted Boltzmann Machine achieves competitive performance on the CIFAR-10 object recognition task.
منابع مشابه
Modeling Natural Image Covariance with a Spike and Slab Restricted Boltzmann Machine
In this work we introduce the spike and slab RBM. The model is characterized by having both a real valued vector: the slab, and a binary variable: the spike, associated with each unit in the hidden layer. The spike and slab RBM possesses some practical properties such as being amenable to Block Gibbs sampling as well as being capable of generating similar latent representations of the data to t...
متن کاملUnsupervised Models of Images by Spikeand-Slab RBMs
The spike and slab Restricted Boltzmann Machine (RBM) is defined by having both a real valued “slab” variable and a binary “spike” variable associated with each unit in the hidden layer. In this paper we generalize and extend the spike and slab RBM to include non-zero means of the conditional distribution over the observed variables given the binary spike variables. We also introduce a term, qu...
متن کاملTexture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions
We apply the spike-and-slab Restricted Boltzmann Machine (ssRBM) to texture modeling. The ssRBM with tiled-convolution weight sharing (TssRBM) achieves or surpasses the state-of-the-art on texture synthesis and inpainting by parametric models. We also develop a novel RBM model with a spikeand-slab visible layer and binary variables in the hidden layer. This model is designed to be stacked on to...
متن کاملLarge-Scale Feature Learning With Spike-and-Slab Sparse Coding
We consider the problem of object recognition with a large number of classes. In order to overcome the low amount of labeled examples available in this setting, we introduce a new feature learning and extraction procedure based on a factor model we call spike-and-slab sparse coding (S3C). Prior work on S3C has not prioritized the ability to exploit parallel architectures and scale S3C to the en...
متن کاملA Spike and Slab RBM Approach to Modeling Natural Images
The spike and slab Restricted Boltzmann Machine (ssRBM) is defined by having both a real valued “slab” variable and a binary “spike” variable associated with each unit in the hidden layer. Earlier work exploring a simple incarnation of the ssRBM model family demonstrated its utility as a model of natural images. In this work, we explore an extension of the ssRBM model – the μ-ssRBM – that inclu...
متن کامل